Multimedia systems integrate multiple sources of electronically encoded audio and video for simultaneous, synchronized presentation over an integrated set of output devices. For example, audio information might be stored on magnetic tape or a compact disc (CD), or obtained directly from a microphone coupled to a digital sampler; video information can originate with a videodisc, CD (possibly, although not necessarily, the same CD containing audio information), a laser disc, or an active digital video camera. A multimedia presentation can also include graphical components, text, and still images, all represented digitally and introduced into the presentation at appropriate times; moreover, in so-called "virtual reality" environments, a presentation can include "haptic" or tactile information obtained from and imparted to the user through suitable transducer arrangements (such as the well-known data glove).
More generally, the presentation is conveyed to a viewer over output devices capable of converting the various digital representations into their native form. Multimedia output systems can range, for example, from no more than a computer-driven video monitor and associated sound hardware to integrated high-end stereo and digital video projection systems and full-body haptic transducer arrays.
Managing the equipment that permits multimedia presentations to be created from their individual audiovisual components, stored, and later retrieved for playback requires substantial processing power. The multimedia information itself can involve prodigious amounts of electronic data even for relatively short presentation sequences, all of which must be handled in a manner that preserves temporal interrelationships. Ordinarily, computational responsibility for the various routing and control aspects of multimedia presentations is shared between a high-level multimedia application program, with which the user interacts to compose, edit, and view the presentation, and a multimedia operating system that performs basic, low-level operations such as data retrieval and output-device designation. This, architecture simplifies the creation of multimedia applications by relieving programmers of the need to repeatedly specify routine data-handling operations at the computer system's most basic level; the multimedia operating system acts as a bridge between high-level, user-oriented application tasks and low-level input/output (I/O) and processor commands.
Recognizing the growing importance of multimedia applications, designers of basic computer operating sysems have upgraded their systems to include specific facilities for multimedia support. These systems effectively have "built-in" multimedia architectures that programmers can invoke as appropriate. Representative of such systems is the IBM Multimedia Presentation Manager/2.TM. (MMPM/2.TM.) supplied by International Business Machines Corporation (IBM) as an extension to its OS/2.RTM. operating system.
Among the tasks performed by MMPM/2 is capture and formatting of raw presentation data from real-time peripheral devices such as cameras and digitizers. Typically, a user records audio and video with a traditional transducer device such as a microphone or video camera, which converts the sound or light-intensity pattern into an analog signal. This signal may be processed into a stream of digital values by a dedicated hardware circuit, which repeatedly "samples" the magnitude of the analog signal at a rapid rate, or left as an analog recording. If the data is digitized, the multimedia operating system handles the tasks of capturing the data stream and storing it as a data file in a format that preserves the temporal or sequence characteristics of the recorded sounds and scenes. Ideally, the multimedia operating system is capable of managing analog as well as digital data.
Key to proper presentation of the various media components is synchronization. Unless the components are presented in a temporally aligned manner, the integrated multimedia output will not make sense to a viewer. Various techniques have been devised to synchronize purely digital data; see, e.g., U.S. Pat. No. 5,333,299 (Koval et al., SYNCHRONIZATION TECHNIQUES FOR MULTIMEDIA DATA STREAMS). The difficulties increase substantially, however, when some of the data is digital and some analog. Asynchronous flow can easily be detected among streams of digital data by direct comparison of streaming rates and cumulative data transfer, and synchronization mismatches corrected by bitwise data manipulations. These techniques, readily implemented at the operating system level and which operate essentially independently of data source, cannot be applied to analog data, since typically the only temporal control that can be exerted on an analog playback device relates to its play speed. Furthermore, the progress of an analog presentation cannot be monitored by digital techniques such as bit-counting. Synchronizing analog and digital information, therefore, ordinarily requires substantial application involvement and knowledge of the particular characteristics of each analog playback device.